This notebook enables up to view raster outputs (.tif and .nc) on the cluster without needing to transfer files
Basic Parameters: PrintDate = 2023-05-28 brdf_dir = /home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf data source = stac gridCell = 4057 index_dir = /home/downspout-cel/paraguay_lc/stac/grids/004057/brdf_ts/ms/evi2 out_dir = /home/klwalker/data/tmp Plotting Parameters: Viewband = nir imageType = Sentinel plotYr = 2021 plotDay = 100
closest downloaded Sentinel image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/sentinel2/S2A_21JYM_20210410_0_L2A.tif with file size 48855339
closest Sentinel brdf is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf/L3B_S2A_brdf21JYM0L2A_20210410_20230217_C01_coreg.nc with file size 32449684
closest downloaded Landsat image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC08_L2SP_224078_20210403_02_T1.tif with file size 108342256
closest downloaded Landsat5 image is:
there are no Landsat5 images for target year 2021
closest downloaded Landsat7 image is:
there are no Landsat7 images for target year 2021
closest downloaded Landsat8 image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC08_L2SP_224078_20210403_02_T1.tif with file size 108342256
closest downloaded Landsat9 image is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC09_L2SP_224078_20211103_02_T1.tif with file size 108324985
closest Landsat brdf is:
/home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf/L3A_LC08_brdf22407802T1_20210403_20230217_C01.nc with file size 33322059
closest smoothed evi2 index image is:
/home/downspout-cel/paraguay_lc/stac/grids/004057/brdf_ts/ms/evi2/2021100.tif
/home/downspout-cel/paraguay_lc/stac/grids/004057/brdf_ts/ms/evi2/2021100.tif with file size 8858221
plotting nir band of image /home/sandbox-cel/paraguay_lc/stac/grid/004057/brdf/L3B_S2A_brdf21JYM0L2A_20210410_20230217_C01_coreg.nc this is in .nc format epsg:8858 ['blue', 'green', 'nir', 'red', 'swir1', 'swir2'] Coord range is: y: -3279710.0--3259500.0. x: 3198730.0-3218940.0 The no data value is: nan
plotting nir band of image /home/sandbox-cel/paraguay_lc/stac/grid/004057/sentinel2/S2A_21JYM_20210410_0_L2A.tif this is in .tif format The no data value is: 32768.0 min and max nir are 327.0 - 32768.0
plotting nir band of image /home/sandbox-cel/paraguay_lc/stac/grid/004057/landsat/LC08_L2SP_224078_20210403_02_T1.tif this is in .tif format The no data value is: 32768.0 min and max nir are nan - nan
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).
/home/klwalker/Jupyter/LUCinSA_helpers/LUCinSA_helpers/notebooks/../LUCinSA_helpers/file_checks.py:86: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_lan['date'] = df_lan.index.map(lambda x: int(x.split('_')[3][:8]))
/home/klwalker/Jupyter/LUCinSA_helpers/LUCinSA_helpers/notebooks/../LUCinSA_helpers/file_checks.py:88: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
df_sen['date'] = df_sen.index.map(lambda x: int(x.split('_')[2][:8]))
| id | dl | beforeDB | brdf_id | brdf_error | brdf | bandpass | coreg | shift_x | shift_y | coreg_error | dldate | numpix | redownload | error | skip | skip_reason | sensor | date |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LC09_L2SP_224078_20211223_02_T1 | /home/sandbox-cel/paraguay_lc/stac/grid/004057... | True | L3A_LC09_brdf22407802T1_20211223_20230217_C01.nc | NaN | True | False | False | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | lc09 | 2021-12-23 |
| LC08_L2SP_224078_20211028_02_T1 | /home/sandbox-cel/paraguay_lc/stac/grid/004057... | True | L3A_LC08_brdf22407802T1_20211028_20230217_C01.nc | NaN | True | False | False | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | lc08 | 2021-10-28 |
| LC08_L2SP_224078_20210926_02_T1 | /home/sandbox-cel/paraguay_lc/stac/grid/004057... | True | L3A_LC08_brdf22407802T1_20210926_20230217_C01.nc | NaN | True | False | False | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | lc08 | 2021-09-26 |
| LC08_L2SP_224078_20210129_02_T2 | /home/sandbox-cel/paraguay_lc/stac/grid/004057... | True | L3A_LC08_brdf22407802T2_20210129_20230217_C01.nc | NaN | True | False | False | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | lc08 | 2021-01-29 |
| LC08_L2SP_224078_20210113_02_T1 | /home/sandbox-cel/paraguay_lc/stac/grid/004057... | True | L3A_LC08_brdf22407802T1_20210113_20230217_C01.nc | NaN | True | False | False | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | lc08 | 2021-01-13 |
--------------------------------------------------------------------------- KeyError Traceback (most recent call last) File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/core/indexes/base.py:3802, in Index.get_loc(self, key, method, tolerance) 3801 try: -> 3802 return self._engine.get_loc(casted_key) 3803 except KeyError as err: File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/_libs/index.pyx:138, in pandas._libs.index.IndexEngine.get_loc() File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/_libs/index.pyx:165, in pandas._libs.index.IndexEngine.get_loc() File pandas/_libs/hashtable_class_helper.pxi:5745, in pandas._libs.hashtable.PyObjectHashTable.get_item() File pandas/_libs/hashtable_class_helper.pxi:5753, in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: 'file_path' The above exception was the direct cause of the following exception: KeyError Traceback (most recent call last) Cell In[39], line 5 2 end = datetime.datetime.strptime('{}-12-31'.format(single_plot_params['plot_yr']),'%Y-%m-%d').date() 4 dfslice = processing_db[start:end] ----> 5 brdf_file_list = dfslice['file_path'].tolist() 6 dl_file_list = dfslice['orig_file_path'].tolist() 7 #print('There are {} files in file list'.format(len(file_list))) File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/core/frame.py:3807, in DataFrame.__getitem__(self, key) 3805 if self.columns.nlevels > 1: 3806 return self._getitem_multilevel(key) -> 3807 indexer = self.columns.get_loc(key) 3808 if is_integer(indexer): 3809 indexer = [indexer] File ~/miniconda3/envs/venv.lucinla38_jupyter/lib/python3.8/site-packages/pandas/core/indexes/base.py:3804, in Index.get_loc(self, key, method, tolerance) 3802 return self._engine.get_loc(casted_key) 3803 except KeyError as err: -> 3804 raise KeyError(key) from err 3805 except TypeError: 3806 # If we have a listlike key, _check_indexing_error will raise 3807 # InvalidIndexError. Otherwise we fall through and re-raise 3808 # the TypeError. 3809 self._check_indexing_error(key) KeyError: 'file_path'